In the past several years, electronics researchers working in the field of analog neural networks and associative memories have attempted to duplicate the logic performed by the human brain. Circuit models have been proposed which provide for both learning (e.g., a programming mode) and decision-making (e.g., recognition, associative memory, etc.).
One category of tasks that neural networks are particularly useful in performing are recognition tasks. That is, neural networks may be utilized to analyze data and identify which features are present. This involves matching templates and expected features to the data and finding the best fit. In pattern matching problems the same stored patterns are repeatedly compared to different input patterns using the same calculation. Various measures of the relative match between the patterns may then be computed.
One measure of the difference between two binary patterns is referred to as the "Hamming distance". (A binary pattern or binary vector is defined in this application to be a string of 1's and 0's, i.e., bits in which the sequence of the bits is meaningful.) Mathematically, the Hamming distance between two patterns is the number of the bits which are different in the two patterns in corresponding bit positions. Of course, both patterns must have the same number of bits for the Hamming distance to have any real meaning, as this provides one-to-one correspondence between bit positions. A semiconductor cell useful in performing pattern matching in a neural network is disclosed in co-pending application entitled "EXCLUSIVE-OR Cell For Pattern Matching Employing Floating Gate Devices", Ser. No. 325,380, filed Mar. 17, 1989 and assigned to the assignee of the present invention; which application is herein incorporated by reference.
Implementing large neural networks in VLSI architecture presents several technical problems which do not normally arise in smaller circuit configurations. By way of example, in an ordinary memory product, only a portion of the array (corresponding to a specific address, i.e., one cell per output bit) is active during a read cycle. In other words, the output is entirely digital in nature and does not depend on the sum of the output currents of each of the cells within the array. Conversely, a neural network may have all of the cells in its entire array active at the same time, providing a large total output current.
As described in the above-referenced application, each neural network matching cell (e.g., EXCLUSIVE-OR cell) may produce a current which is typically summed in an analog manner along one or more column lines. These column lines must be capable of handling very small currents (when none of the cells contribute current) as well as very large currents (for situations in which many or all of the cells are contributing current).
Very large column currents can lead to electromigration of the metal which is used to fabricate the column lines within the array. Electromigration is a leading cause of reliability-type failures. If an attempt is made to lower the read current along the current summing lines, signal interference (i.e., noise) may degrade the accuracy of the computation. The amount of current that any active cell may contribute to the total column current, of course, depends on the programming (i.e., learning) state, as well as the input vector applied.
Since relatively high current can flow on each summing line for a large array, ohmic voltage drop and electromigration presents a serious problem for very large neural network arrays. (The former generates signal interference which degrades the accuracy of the computation while the later gives rise to reliability problems).
As will be seen, the present invention solves these problems by use of a novel neural network architecture. According to the present invention, a neural network is subdivided into a number of smaller blocks. Each block comprises an array of pattern matching cells such as those described in the above-mentioned co-pending application. The cells are arranged in columns along one or more local current summing lines. These lines provide a means of adding together the current contributions from each cell to produce a measure of the relative match between a binary input pattern applied to the block and a binary weight pattern stored within the cells of the array.
In addition, each column line within a block is coupled to a global summing line through a specialized coupling network. The global summing lines add the associated local column currents to produce a measure of the Hamming distance for a plurality of blocks. The type of architecture described in this application is referred to as a leveled summing scheme with blocked array (LSBA) and provides an elegant solution to the problem of large neural network implementation.
Other prior art known to applicant which is believed pertinent to this application includes U.S. Pat. No. 4,760,437 of Denker et al.; U.S. Pat. No. 4,660,166 of Hopfield; U.S. Pat. No. 4,782,460 of Spencer; U.S. Pat. No. 4,773,024 of Faggin et al.; U.S. Pat. No. 4,802,103 of Faggin et al.; and "A Pipelined Associative Memory Implemented In VLSI" by Clark et al., IEEE Journal of Solid State Circuits, Vol. 24 No. 1, pp. 28-34, February 1989.